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ABSTRACT 


Successful commercial companies understand that customers are the real experts 
with regard to their products and services. Bringing customer experiences right into the 
design shop allows development of best-selling commercial products and services. 
Companies such as L.L. Bean, Inc. immerse themselves in their customer’s experiences 
during new product development. They travel to their customer’s location and listen to 
them face to face to get the best possible input for essential product requirements and 
new design ideas. 

Currently, most military C4I systems product development does not make 
effective use of customer input. Systems are developed and fielded in accordance with 
Department of Defense regulations that provide insufficient mechanisms for users to 
influence product requirements and design. C4I system program managers need 
additional tools to obtain and translate user needs into system requirements and designs. 

Harvard Business School has developed an educational program aimed at 
redesigning product/service development based on the L.L. Bean model. This thesis will 
apply the tenets of that program to submarine C4I systems development and identify 
obstacles to and lessons learned from its application to military product development. 
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I. INTRODUCTION 


A. BACKGROUND 

Customer-driven product development is a key component of successful commercial 
business practices in today’s environment. Organizations such as L.L. Bean, Incorporated 
and Ford Motor Company integrate and ingrain the voice of the customer into their product 
development culture. Ford’s Consumer Relationship Process is defined by “A sustained 
relationship with the consumer through the development of consumer insight, which is used 
in the development and delivering of products and services” (Renaud, 2000, p.8). This type 
of customer focus can benefit all organizations, both private and government. Ability to 
meet the needs cf the customer can be the difference between success or failure in the 
commercial marketplace and in government systems development. Unfortunately, many 
barriers exist to successfully integrating the voice of the customer into military systems 
development. The Government Accounting Office (GAO) has issued a series of reports on 
the success that leading commercial firms have had in significantly reducing the time and 
money needed to develop new and more sophisticated products - the same results that the 
Department of Defense (DoD) desires. Significant cost and schedule increases, as well as 
significant user dissatisfaction, can be traced to an insufficient definition of customer 
requirements prior to program management decisions. 

The basic process for formulating product requirements is the same for commercial 
firms and DoD. Understanding the customer’s expectations is the first step for both. These 
expectations must then be translated into product requirements, including the functions. 


1 



characteristics, reliability and maintainability it must possess. It is not unusual that the first 
understanding of customer expectations exceeds the capacity of the product developer given 
the current state of technology as well as limited resources and time available. Table 1 
highlights the contrasts between the interests of the customer and product developer. 


Customer wants _ 

Performance: what the product should do. 
For example, what an aircraft’s speed, 
range, fuel economy, reliability and other 
features should be. 

Timing: when the customer wants the 

product. 


Pricing: what the product will cost. The 
customer must be able to afford the 
product. 


Product Developer’s resources _ 

Technology: the technology that is needed 
to make the product function to a level 
necessary to meet the customer’s wants. 

Schedule: the amount of time that is 

needed to develop, design, test and 
manufacture the product. 

Investment funds: the capital that is 

needed to pay for development, test, and 
manufacture of the product. 

Expertise: the capabilities of the product 
developer in terms of engineering 
expertise, manufacturing capabilities and 
production. 


Table 1: Customer and Product Developer Interests in a Product’s Developments 
(From: GAO Report 01-288, March 2001) 

All product developments attempt to match customer expectations with available 
resources in order to define the product. The matching process leads to close scrutiny of the 
customer expectations and developer’s resources, resulting in a set of product requirements 
that lepresent an agreement between the customer and developer on the product’s planned 
performance, cost and schedule. This basic requirements process is depicted in figure 1 
below. 
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Figure 1: The Requirements Process (From: GAO Report 01-288, March 2001) 

Translation of customer expectations into a set of specific product requirements 
includes information gathering, analysis and negotiation. In commercial product 
developments, these negotiations occur directly between the customer and product developer 
before resources are committed to the product development. During this process, the general 
customer expectations are reduced to a set of performance requirements that are achievable 
within the developer’s available resources and still meet the customer’s needs. Two-way 
communications occur between the customer and product developer during this process. For 
example, during new commercial aircraft development, the customer may desire a certain 
speed in order to enhance passenger revenue, but the developer determines that the time and 
resources available are insufficient to achieve that speed. Negotiation then ensues until both 
parties agree upon a speed requirement that satisfies the customer need within the available 
development resources. 

DoD employs a more complex requirements process involving communication 
between at least four players. Separating the customer (an organization in the fighting 
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forces) and the product developer (a defense contractor), is the user representative 
(organization lepresenting the customer in product requirements negotiations) and the DoD 
Program Manager (essentially representing the product developer in requirements 
negotiations) (GAO Report 01-288, pp 18-19). Figure 2 depicts the interplay between the 
major players in the commercial and DoD requirements processes. 


Commercial example: requirement for a new airliner 


Customei 


Product Developer 

Airline company 

-► 

Aircraft manufacturing firm responsible 

desiring a new 


for developing both the product 

passenger aircraft. 


requirements and the product itself. 


DoD example: requirement for a new fighter aircraft 



Figure 2: Commercial and DoD Organizations Involved in Requirements 
Setting Processes (From: GAO Report 01-288, March 2001) 

Both commercial and defense organizations are concerned with the cost, schedule and 
performance aspects of a product’s development, and these concerns are reflected in the final 
set of specific product requirements. In the defense process, however, the ultimate product 
developer usually has little input into the requirements development process. Formal product 
requirements, in the form of an Operational Requirements Document, are provided to the 
prime contractor at the start of the actual product development cycle. 
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B. PURPOSE 


The purpose of this thesis is to evaluate the applicability of a specific commercial 
product development process used by L.L. Bean, Incorporated to define customer 
requirements to military C4I systems development. The most effective means of 
accomplishing this is by applying the tenets of that process to the development of specific 
military C4I systems. Submarine Exterior Communications Systems have been selected as 
the candidate systems to evaluate this approach as a result of the author’s position within the 
Submarine Communications Program Office and extensive background in submarine 
operations and communications. This research includes interviews with submarine officers 
and enlisted communications personnel currently serving aboard fleet units. The specific 
interview questions are developed using the L.L. Bean model. 

The results of this analysis will be used to enhance the development of the submarine 
C4I systems investigated in addition to assessing the applicability of the process to military 
C4I system development. Use of the L.L. Bean product development process should allow 
for enhanced communication of customer expectations directly to program office personnel, 
thus streamlining the requirements negotiation process depicted in Ligure 2 above. 

C. RESEARCH QUESTIONS 

The following research questions fall into two categories. The first category of 
questions will be answered primarily through review of current literature. The second 
category will be answered by conducting interviews with submarine C4I system users using 
the L.L. Bean method and analyzing the results. 
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1. Research Questions Answered by Literature Research 

Before applying the L.L. Bean model to submarine C4I system development, an 
investigation of current best practices in commercial customer-driven product development 
and military adaptation of these commercial best practices is required. Potential benefits and 
drawbacks of customer-driven military C4I systems product development should also be 
identified and researched. The following questions address these areas: 

1. What are the current best practices in customer-driven commercial product 
development? 

2. How have commercial best practices in customer-driven product development 
been adapted for military use? 

3. What are the key elements to customer-driven product development (commercial 
or military)? 

4. What are the benefits of customer-driven military C4I systems product 
development? 

5. What are the drawbacks of customer-driven military C4I systems product 
development? 

6. What are the special considerations required for customer-driven military C4I 
systems product development? 

2. Research Questions Answered by Interview Results 

The crux of this thesis is based on applying the L.L. Bean product development 
interview technique to submarine C4I systems development. The following questions were 
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used in adapting that technique to gather submarine officer and enlisted communications 
personnel expectations with regard to submarine C4I systems. 

1. What type of interview questions allows users to convey their experiences and 
ideas regarding military C4I systems development and use? 

2. What is the best way to conduct interviews with users to enhance military C4I 
systems product development? 

3. What is the best way to analyze data collected during user interviews? 

4. How can user statements during interviews be translated into military C4I system 
product requirements? 

5. How can validated product requirements be translated into product design ideas? 

6. How can product design ideas be incorporated into military C4I systems product 
development? 

D. BENEFITS OF STUDY 

The benefits of this study are twofold. First and foremost, the applicability of the 
L.L. Bean interview techniques for capturing the voice of the customer will be evaluated in a 
military systems development environment. If successful, this technique could provide 
defense program managers with a powerful tool to help define customer expectations and 
negotiate with users to develop product requirements that satisfy user needs within the 
constraints of the product development resources and timeframe available to the program 
manager. 

Secondly, the Submarine Communications Program Manager will benefit by 
receiving direct user input regarding the performance and suitability of submarine C4I 
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systems onboard fleet units today. This information should enhance the program manager’s 
decision-making ability regarding planned product development efforts, ultimately resulting 
in improved C4I products and systems and enhancing submarine force connectivity in the 
future. 

E. SCOPE AND METHODOLOGY 

1. Scope 

The scope of this thesis is divided into five phases. The first phase involves a review 
of commercial product development best practices and their application to military systems. 
This phase is important to provide an understanding of successful product development 
practices currently in use. It will provide a baseline assessment for how well customer 
desires are captured and included in both the commercial and military product development 
process. 

The second phase consists of a review of the Harvard Business School’s Redesigning 
Product/Service Development program based on the L.L. Bean, Incorporated model. This 
phase is necessary to understand the L.L. Bean method of incorporating the vrice of the 
customer into product development and to begin adapting that method for military C4I 
systems development. 

The third phase consists of application of the tenets of the Harvard Business School’s 
Redesigning Product/Service Development program to submarine C4I systems product 
development. Gathering data on customer expectations from submarine operators through 
targeted interview questions developed during phase two is the focus of this phase. 
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The fourth phase entails an in-depth analysis of submarine customer interviews 
conducted in accordance with the L.L. Bean model. The voice of the submarine customer 
can thereby be determined and used to enhance submarine C4I systems development. An 
assessment of the applicability of the L.L. Bean method to military C4I systems development 
will also be performed. 

The fifth and final phase consists of translation of customer-expressed needs to 
submarine C4I systems product requirements development and design. This phase will allow 
program mangers developing submarine C4I systems to make decisions and trade-offs 
regarding cost, schedule and performance of developments under their cognizance. 

The end result of this research is to test the L.L. Bean product development model in 
a military C4I systems environment. It will conclude with recommendations for future 
applications of the model. 

2. Methodology 

The methodology used in this thesis research consists of the following steps: 

1. Conduct a literature review of books, magazine articles, CD-ROM systems, and 
other library information resources on customer-driven product development best 
practices. 

2. Conduct a thorough review of previous research. 

3. Conduct a review of the Harvard Business School’s Redesigning Product/Service 
Development program based on the L.L. Bean model. 

4. Examine the current military C4I systems product development model. 
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5. Develop submarine C4I system eustomer interview questions using the Harvard 
Business Sehool program tenets. 

6. Conduet submarine C4I system eustomer interviews in aeeordanee with the L.L. 
Bean model. 

7. Thoroughly analyze submarine C4I system eustomer inputs and translate them 
into system requirements and design ideas. 

8. Evaluate effieaey of the Harvard Business Sehool program as it relates to military 
C4I systems development. 

F. ORGANIZATION OF STUDY 

This study eonsists of six ehapters, whieh deseribe eurrent best praetiees in 
eommereial produet development and their applieation to military systems, the Harvard 
Business Sehool’s Redesigning Produet/Serviee Development program based on the L.L. 
Bean model, and an applieation of it to submarine C4I systems development. Chapter I 
provides a brief introduetion and summary of this thesis, ineluding an assessment of the need 
for improved methods of eolleeting eustomer desires and expeetations in military systems 
development. Chapter II eonsists of a literature review on best praetiees in eommereial 
produet development and their applieation to military systems. The L.L. Bean Produet 
Development model is also deseribed in detail here. 

In Chapter III, the researeh methodology is deseribed. This ehapter explains the steps 
required in applying the L.L Bean model to submarine C4I system produet development and 
the speeial eonsiderations required. Chapter IV provides a detailed analysis of the results of 
submarine eustomer interviews. Chapter V provides an evaluation of the Harvard Business 
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School’s Redesigning Product/Service Development program application to submarine C4I 
systems product development. Finally, Chapter VI summarizes this research, provides 
lessons learned and recommends further research areas. 


11 



THIS PAGE INTENTIONALLY LEET BLANK 


12 



II. LITERATURE REVIEW 


A. INTRODUCTION 

In product development, satisfying customer needs is recognized as a key competitive 
success factor in today’s commercial marketplace. Product development teams consisting of 
engineers and designers must be able to extract a high degree of detail regarding customer 
expectations in order to be successful. A laser-like focus on the customer is necessary to 
determine the best possible answers to the following product development relevant questions: 
(1) What exactly do customers value? (2) From these values, what are the product 
requirements that will achieve the highest possible level of customer satisfaction? (Renaud, 
2000). The voice of the customer can provide a new product idea or suggest innovations for 
an existing product. Effectively engaging the customer voice enables a company or 
government program office to better understand how customers use their products. A 
product’s success (or failure) can be a direct result of the company/program office’s ability to 
understand and satisfy the voice of the customer. 

To date, the degree of improvement in new product development methods has been 
higher in the commercial sector than the defense sector. The commercial world has 
integrated the customer voice throughout the organization as the primary means to improve 
product development. In DoD, institutional barriers can hinder widespread integration of the 
voice of the customer in product development. Government product developers tend to focus 
on development process and methods, providing less interaction with users during product 
development. 
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This chapter examines several current best practices in commercial product 
development and their application to military systems. It discusses how successful 
companies integrate the voice of the customer into their product development strategies, and 
how government program offices could benefit from similar strategies. The chapter then 
focuses on the Harvard Business School’s Redesigning Product/Service Development 
program, describing how L.L. Bean, Incorporated integrates the voice of the customer into 
their product development process and how to apply the same principles to military 
product/service development efforts. 

B. COMMERCIAL BEST PRACTICES 

Many different techniques are employed by commercial organizations to obtain and 
use the voice of the customer during the product development process. Often, the method 
used to gain insight into the customer voice depends on the company’s specific objective and 
type of information desired. Companies seeking meaningful feedback from a relatively large 
group of customers at once may employ the focus group method. Survey methods can 
provide insight on how customers rate specific product features. Worldwide Web-based 
methods, such as the Sloan School of Management’s Information Pump have also been 
developed. Finally, one-on-one interviews can be used to provide detailed customer 
expectation data from individual users. Each of these methods has distinct strengths and 
weaknesses as discussed below. 

I. Focus Groups 

Product development practices using focus groups bring together multiple users in a 

single setting to gain insight from the group as a whole. A moderator is usually employed to 
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lead group discussions directed at obtaining customer expectations with regard to product 
features under consideration by the company’s product development team. Focus group 
feedback can provide a good indication regarding how the larger customer population will 
likely react, allowing the company to evaluate whether or not they should proceed with a 
particular idea. A significant advantage of the focus group technique is that it allows for a 
variety of perspectives to be advanced and discussed in an open forum, often leading to frank 
and constructive product development inputs. However, the focus group technique can suffer 
if one or more individuals dominate the discussion and influence the overall group response. 
In most cases, “the focus group can provide helpful insight into the voice of the customer” 
(Mastronardi, 2001, p. 17). 

2. Customer Surveys 

Surveys can be provided to either a target set of customers or randomly to gain 
insight regarding how customers might perceive product or service features. Many types of 
surveys and methods of administering them are currently in use. Telephones, the Internet, 
mailings and face-to-face surveys at the point of sale can all be effective in capturing the 
voice of the customer. Advantages of the survey method include: (1) detailed data regarding 
customer perceptions of a product or service can be obtained and (2) statistically relevant 
metrics can be developed and displayed conveniently for decision-makers. The primary 
disadvantages of the survey method include the potential for a high numbers of non¬ 
responses and customer misunderstanding of questions leading to an incorrect assessment of 
customer product desires. Surveys can be useful in providing customer voice information to 
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companies, but their use is generally limited to more targeted roles than other methods 
(Mastronardi, 2001). 

3. Information Pump 

Drazen Prelec (Professor of Management Science, Sloan School of Management, 
Massachusetts Institute of Technology) has developed a method called the Information Pump 
focusing on providing incentives for collecting customer information. These incentives are 
integrated into a web-based information system for data collection relevant to the product 
design process. This incentive-based approach has the advantage of allowing for consistent 
motivation and effort by the customer, and the incentives clearly communicate the 
information desired from customers. By allowing customers to monitor their progress, 
improved responses can be obtained over time. This approach also identifies the most 
responsive customers who can then be used for later product development efforts. Two 
different approaches can be used to create incentives with customers: (1) linking 
compensation to the outcome of the project and (2) enabling the comparison of responses 
from two customers simultaneously via the Internet. The Information Pump can provide the 
voice of the customer via the Web and enable customers to compare and contrast desirable 
product features via web-based interaction with one another. This method also has the 
distinct advantage of incorporating a scoring method for qualitative and quantitative 
feedback (Mastronardi, 2001). 

4. One -o n-One Interviews 

Direct interviews with targeted customers provide unfiltered feedback to companies 

regarding customer expectations that can be extremely useful to product development teams. 
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Questions posed to the interviewees are often open-ended to allow the interviewee to take the 
lead in discussions. An interview guide, developed beforehand, allows the interviewer to 
keep the discussions on track with the research objectives. The voice of the customer may 
express a product need, an area where a specific product or service fell short of expectations 
or even a product feature that created customer delight. One-on-one interviews can be 
invaluable in helping companies determine how their products are actually used and how 
they fit into their customers’ lifestyle. The success of one-on-one interviews in revealing the 
voice of the customer is directly dependent on the structuring of interview questions to meet 
the desired objectives (Mastronardi, 2001). The Harvard Business School’s Redesigning 
Product/Service Development method relies strongly on one-on-one interviews to understand 
customer expectations and integrate the voice of the customer into the product development 
process. 


C. GOVERNMENT ADOPTION OF COMMERCIAL BEST PRACTICES 

For several years, the Government Accounting Office (GAO) has been 
recommending the adoption of commercial best practices for DoD product development. 
Their opinion is “the best practices of leading commercial firms can be used to improve the 
development of technology and weapon systems in DoD” (GAO Report 99-116, p. 1). The 
best practices model advocated by GAO is based on developing knowledge to answer the 
basic question of how a capability can best be provided to the customer. Understanding the 
voice of the customer is a crucial first step in developing the requisite knowledge base 
advocated by the GAO. DoD regulations also recognize the importance of customer input. 
“The Defense acquisition and requirements communities shall maintain continuous and 
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effective communications with each other and with the operational user. The objective is to 
gain a sound understanding of the users’ needs and to work with them to achieve a proper 
balance among cost, schedule, and performance considerations.” (DoD Instruction 5000.1, 
Paragraph 4.2.2). To help foster the required “effective communications” with operational 
users, acquisition program offices could employ the Harvard Business School’s Redesigning 
Product/Service Development method being evaluated herein. 

GAO has also identified several successful DoD initiatives where commercial best 
practices have been adopted to enhance military system development. Examples include 
maturing technologies prior to incorporating them into programs of record (e.g., VIRGINIA 
(SSN 774) Photonics Mast, the Advanced Amphibious Assault Vehicle propulsion 
technology, and the Air Force’s Integrated High Performance Turbine Engine Technology 
Program), emphasizing cost as an independent variable and utilizing integrated product teams 
to manage system development (GAO Report 99-116). Government program office success 
in adopting some commercial best practices indicate that others, such as those used to obtain 
and integrate the voice of the customer into product development, can be effectively adapted 
to enhance DoD processes. 

D. HARVARD BUSINESS SCHOOL’S REDESIGNING PRODUCT/SERVICE 

DEVELOPMENT PROGRAM 

Harvard Business School has developed a program titled “Redesigning 
Product/Service Development” based on the E.E. Bean, Incorporated process for obtaining 
the voice of the customer. This program provides a toolkit to guide product developers in its 
implementation in their own setting. The toolkit contains a video introducing the benefits 
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and steps of customer-driven product or service development, using L.L. Bean, Inc., as an 
example, a blueprint detailing how to apply the video techniques to other product or service 
development challenges and worksheets to facilitate organization of the customer-driven 
product development work. The material in the toolkit is based on the writing and research 
of Professor David A. Garvin. This program is described in detail in the paragraphs below. 

1. L.L. Bean Model 

The key points of the Harvard Business School customer-driven product development 
model can be summarized as follows (Garvin, 2000): 

a. Customers are the real product or service experts. 

b. Developing the best products or services requires understanding and bringing 
customer experiences right into the design shop. 

c. Don’t isolate product developers from customers with outside market researchers 
or focus group one-way mirrors. Immerse developers in customer experiences. 

d. Go right to the customer’s turf. 

e. Listen to customers face to face. 

f Watch customers actually using products or services. 

g. Translate what you see and hear into the customer’s essential product or service 
requirements 

h. Develop product or service designs that precisely meet those essential 
requirements 

i. Highlight this customer-driven approach in your marketing. 
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The video provided in the Harvard Business School program toolkit depicts actual 
L.L. Bean product developers applying these key points to the development of new products. 
The blueprints and worksheets provide a framework for adapting these tenets and methods to 
other products and services. 

2. Criteria for Field Testers 

The early stages of customer-driven product or service development rely on a very 
small group of customers (approximately 16-20). If properly selected, this group can be used 
again and again to help develop new products or services and improve existing ones. Thus, 
field tester selection is a critical part of the process. The goal is to form a group of field 
testers who are: (1) experienced users of the target product/services, (2) demographically 
diverse, and (3) articulate, honest communicators. The Harvard Business School program 
begins with the definition of field tester selection criteria, and then applies those criteria to 
select field testers from the customer base. This process proceeds in the following steps 
(Garvin, 2000): 

a. Definition of users versus purchasers - although it may be useful to talk with 
purchasers of products and services, it is essential to listen to and watch customers who 
actually use the products and services. For example, parents purchase toys, but children are 
the actual users of toys. 

b. Definition of experienced users - only about 16-20 customers will become field 
testers, so it’s important to choose them from the most experienced product/service users. 
For example, experienced users may be defined as those customers who use X quantity of 
products or services within Y period of time. 
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c. Definition of demographic categories from which diverse representation is needed 
among experienced users - ensuring diversity among experienced users will provide a 
broader range of perspectives regarding the products or services. For example, demographic 
categories can be defined by occupation, gender, age, etc. 

d. Definition of common characteristics for experienced, demographically diverse 
users - the most valuable, experienced, demographically diverse users should have certain 
characteristics in common. For example, L.L. Bean ensures its field testers are articulate 
communicators, analytical in comparison of their products, honest and unbiased, and timely 
in providing feedback. 

3. Field Tester Interviews 

The field tester interview process described in the Harvard Business School’s 
Redesigning Product/Service Development program is based on open-ended interviewing 
techniques that have roots in anthropology (see Schwartzman, pp. 58-60, 66). Two key areas 
of focus for the field tester interviews include the interview questions and the interview pairs. 

The interview questions should enable the field testers to convey the depth of their 
experiences with the product/service by vividly describing what happens and how they feel 
when using them, explaining what problems they’ve had in using them, and expressing their 
excitement regarding them. The questions must be broad and open-ended to promote in- 
depth answers, be non-leading so as not to suggest any particular answer, explore 
comparison’s with other products/services and take no more than an hour of the field tester’s 
time. The final list of question should be narrowed down to no more than five or six. L.L. 
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Bean uses the following types of questions when interviewing hunters who use their boots 
(Garvin, 2000, p. 29): 

- “Tell me your best hunting story.” 

- “Describe a situation in which your hunting footwear let you down.” 

- “If you could design your own custom hunting boots, what would they be like?” 

- “What haven’t I asked you about your footwear that you’d like to discuss?” 

The Harvard Business School program also stresses the need to conduct the 
interviews using a pair of interviewers with distinct roles - an interviewer and a recorder. 
The interviewer encourages field testers to describe their experiences and feelings to say 
what is on their minds, listens attentively, never interrupts and redirects the interviewee only 
when absolutely necessary. For example, the interviewer might ask the field tester to say 
more about his or her experiences if the tester had jumped to product/service solutions prior 
to providing a clear picture of their specific experience. The recorder remains silent and 
attentive, writes down exactly what the field tester says, doesn’t interpret, filter or summarize 
the field tester’s words, and asks for clarification if he doesn’t understand the tester’s 
comments. The method also allows the recorder to discreetly operate a small recording 
device if the field tester is comfortable being recorded, but it notes that most people prefer 
not to be recorded. A recorded transcript of the interview can be useful, but it is not essential 
if the written transcript is carefully maintained (Garvin, 2000). 

4. Interview Data Analysis 

Analysis of the data collected during field tester interviews is conducted in two parts: 
(1) post-interview debriefings and (2) summarizing field tester quotations and descriptions. 


22 



Post-interview debriefings should be conducted as soon as possible after each interview. The 
debriefing allows the interviewer and recorder to review their notes and fill in any gaps. As 
they’re reviewing, they can also begin to capture what the field testers think and feel most 
deeply about their products/services, selecting evocative quotations and vivid descriptions 
and recording them on sheets of paper headed with each interview question. For example, 
L.L. Bean hunter interviews provided the following types of evocative quotations and vivid 
descriptions (Garvin, 2000, p. 33): 

- “walking through wet, swampy areas - water seeping into my boots” 

- “pulling socks way up because the boot rubs and rubs and rubs - hair rubbed right 
off’ 

- “putting on wet, cold boots in the morning - just miserable” 

The next step is to summarize and translate these field tester quotations and 
descriptions using the following steps (Garvin, 2000): 

a. Pool quotations and descriptions - posting each interview question on the wall and 
putting each quotation and description under the appropriate question provides one method to 
accomplish this. 

b. Prioritize and Reduce pooled quotations and descriptions - apply your 
understanding of field tester priorities. 

c. Group prioritized quotations and descriptions - remove the interview questions 
from the wall and rearrange the quotations and descriptions into clusters by themes. 

d. Summarize grouped quotations and descriptions - distill the essence of each cluster 
of field tester quotations and descriptions into a brief statement. For example, the first and 
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third quotation listed earlier in this section regarding L.L. Bean hunting boots could be 
grouped under the summary statement, “Feet wet and cold”. 

5. Translation of Interview Results into Product Requirements and Design 
Ideas 

Each quotation/description summary statement developed from the field tester 
interview analysis is then translated into many field tester requirements. For example, the 
summary statement, “Feet wet and cold” could be translated into two requirements for the 
hunter’s footwear: (1) keep the feet dry and (2) keep the feet warm. All of the product 
requirements thus generated are then pooled, prioritized and reduced, grouped into prioritized 
requirements and then further prioritized and reduced. The final set of prioritized product 
requirements should not contain any redundant requirements and should accurately capture 
the field tester’s priorities. After obtaining a set of valid requirements, each is translated into 
many design ideas. For example, the requirement to keep the feet warm might create design 
ideas for flannel sock liners or insulated boot inserts. All of the design ideas thus generated 
are pooled, prioritized and reduced, grouped into prioritized design ideas and then further 
prioritized and reduced. The final set of grouped design ideas should not contain any 
redundant ideas, should precisely meet the valid requirements and be logistically realistic. 
The final set of design ideas should also be compared to the field tester quotations and 
descriptions to ensure that the ideas and concerns of the experienced, demographically 
diverse product/service users are still driving the development (Garvin, 2000). 
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E. SUBMARINE C4I SYSTEM S OF INTEREST 

Three submarine C4I systems of interest were selected for this study to assess the 
applicability of the Harvard Business School’s Redesigning Product/Service Development 
program and to provide useful feedback to the Space and Naval Warfare Systems Command 
Submarine Communications Program Office regarding planned product enhancements for 
currently fielded systems and future new product developments. Each system and the reason 
for its selection are described briefly in the paragraphs below. 

I. Ultra High Frequency (UHF) Medium Data Rate (MDR) Capability 

Installation of UHF MDR capability on submarines provides a 32 kilobit per second 
(kbps) UHF asymmetric satellite communications capability to support Internet Protocol (IP) 
connectivity, enabling web browsing and e-mail SIPRNET services for the submarine while 
underway. UHF MDR is installed on all fast attack submarines deploying with Battle 
Groups to provide an interim medium data rate IP capability prior to the fielding of 
Submarine High Data Rate antenna. The program office perceives the UHF MDR program 
as an acquisition success story due to it’s relatively low cost, it’s rapid speed to deployed 
capability and it’s importance to future communications concepts cf operations. It is of 
interest to this study and to the Submarine Communications Program Office to determine 
how well the UHF MDR capability was meeting the fleet need for SIPRNET connectivity 
and obtain the voice of the submarine customer perspective regarding needed upgrades for 
the system. 
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2. Mini-DAMA UHF Transceiver 


The AN/USC-42(V) Miniaturized Demand Assigned Multiple Access (Mini-DAMA) 
Program consists of Non-Developmental Item/Commercial Off-The-Shelf (NDI/COTS) 
equipment to provide a communication system that supports the exchange of secure and non- 
secure Battle Group coordination data, tactical data and voice. This system is of interest to 
this study because of the multiple design and logistics problems involved with its fielding on 
operational submarines and the desire to capture the voice of the submarine customer during 
development of planned field changes/upgrades. 

3. Submarine Tactical Data Links 

Submarine Tactical Data Links currently in use onboard operational submarines 
include LINK 11, LINK 16 and Satellite TADIL-J (S-TADIL-J). These systems allow near 
real time transfer of tactical track data (course, speed, identification, etc. of contacts) between 
submarines and other Battle Group units. These systems are of interest to this study to help 
evaluate which of these systems submarine customers actually use and to capture the voice of 
the submarine customer in defining product requirements for systems under currently under 
development to replace these older, legacy tactical data transfer systems. 

F. CHAPTER SUMMARY 

This chapter reviewed commercial best practices used to integrate the voice of the 
customer into product development, along with successful examples of DoD adaptation of 
other commercial best practices. The Harvard Business School’s Redesigning 
Product/Service Development program based on the L.L. Bean, Incorporated, product 
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development model was described in detail. This method emphasizes face-to-face interviews 
with experienced product users, called field testers, using open-ended questions conducted by 
two person interview teams. Interview data are analyzed to provide field tester quotations 
and descriptive phrases that are further developed into product/service requirements and 
design ideas. This chapter also provided a brief description of submarine C4I systems of 
interest that will serve as the test case for application of this commercial product 
development process to military C4I systems development. The remaining chapters of this 
research will describe the application of the Harvard Business School process to the conduct 
of submarine user interviews and development of submarine C4I system product 
requirements and design ideas. 
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III. RESEARCH METHODOLOGY 


A. INTRODUCTION 

The purpose of this research is to apply the Harvard Business School’s Redesigning 
Product/Service Development method of obtaining the voice of the customer to submarine 
C4I systems development, evaluating the applicability of this commercial product 
development process to military systems development. The research methodology is divided 
into three main areas as defined by the Harvard Business School program. The first area 
involves planning the product development interviews to include determining the 
composition of the interview team and developing the questions to be used during the 
interview. Because the goal is to assess the applicability of the Harvard Business School 
program to military systems development, the tenets of that program are followed as closely 
as possible in planning the submarine customer interviews. Careful preparation and 
development of the interview guide will help keep the subsequent interviews focused and 
effective. 

The second major area of research methodology involves selecting field testers from 
the submarine community to hterview. Again, the Harvard Business School process is 
followed as closely as possible and adapted where necessary to include submarine-specific 
criteria. Proper definition of submarine field tester criteria is critical to ensure useful 
feedback is obtained regarding the selected submarine C4I systems of interest and to 
adequately assess the applicability of this commercial product development process to 
military systems. 


29 



The final area of research methodology is the conduct of the interviews themselves. 
In order to analyze the effectiveness of the L.L. Bean product development method in this 
context, feedback must be obtained directly from the selected submarine field testers. The 
interview questions developed in accordance with the Harvard Business School program 
guidelines are used to guide the discussions with the submarine field testers and their 
responses are recorded exactly. In order to accurately capture the voice of the submarine 
customer, the interviews are conducted on their home turf aboard their submarines. The 
results of these interviews help generate conclusions and recommendations regarding the 
applicability of the Harvard Business School program to military systems development and 
provide useful data for the Submarine Communications Program Office efforts to improve 
existing products and develop new ones. 

B. PLANNING PRODUCT DEVELOPMENT INTERVIEWS 

The first step in applying the Redesigning Product/Service Development 
methodology is to plan the product development interviews. Two key areas of initial 
interview planning are the composition of the interview team and the development of 
interview questions used to guide the discussions with field testers. Both of these areas are 
discussed separately below. 

I. Interview Team Composition 

The Harvard Business School’s Redesigning Product/Service Development program 
is quite specific regarding the necessary interview team composition. Two persons make up 
the interview team - one takes the role of interviewer and the other acts as recorder. The 


interviewer facilitates the interview using open-ended questions as a guideline and 
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encouraging field testers to vividly describe their experiences and feelings regarding the 
product being discussed. The recorder remains silent during the interview except to ask for 
clarification when required and writes down exactly what the field tester says. 

For the submarine customer product development interviews, the author acted as 
interviewer, applying his fifteen years of submarine operating experience and 
communications systems expertise to enhance the conduct of the interviews. Selected 
submarine field testers appreciated the interviewer’s detailed understanding of their operating 
environment and assigned missions. Due to the technical nature of the interview discussions, 
it was important that the interviewer be a subject matter expert in the area of submarine 
operations and communications. A group discussion facilitator (Mr. Gary Rossi) from Booz, 
Allen and Hamilton, Inc. was selected to act as the recorder for the submarine customer 
product development interviews. Mr. Rossi also had previous experience as a Naval Officer 
prior to his employment at Booz, Allen and Hamilton. The interview team worked closely 
together to learn and apply the tenets of the L.L. Bean product development process to plan 
and conduct the interviews. 

2. Interview Questions 

The L.L. Bean product development interview process as described in the Harvard 
Business School’s Redesigning Product/Service Development program is also very specific 
with regard to interview question generation. In order to promote detailed, in-depth 
discussions with the submarine field testers, the questions need to be broad and open-ended. 
They must not lead the interviewees to any particular answer and should take no more than 
one hour of the submarine field testers’ time. The goal of the interview questions is to evoke 
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vivid descriptions from the field testers regarding what happens and how they feel when 
using the submarine C4I systems of interest described earlier and explaining what problems 
they have when using them. 

Using these guidelines, the submarine customer product development interview team 
derived the following set of questions that were used to guide the course of all subsequent 
interviews with submarine field testers. These questions were designed to gather information 
regarding the operability and maintainability of the C4I systems of interest installed prior to 
the most recently completed period of deployed operations as well as the submarine customer 
voice regarding the installation process. 

a. Describe the most demanding communications period that occurred during the 
deployment and how well or how poorly the new C4I system installations supported the ship 
during that period? 

b. Describe the most difficult maintenance evolution involving the new C4I systems 
that occurred during the deployment and how well or how poorly the technical 
documentation supported your maintenance efforts. 

c. Describe the training provided during the installation of the new C4I systems and 
how well or how poorly that training prepared you for operating and maintaining the 
equipment during your deployment. 

d. Describe the impact of the new C4I system installation on the ship’s 
routine/schedule. 

e. Are there any other issues or problems with the new C4I systems? Were any 
Casualty Reports (CASREPS) involving the new C4I systems required during the 


deployment? 
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Appendix A provides an example of the interview data sheets developed as part of 
this study and used to record responses from interviewees. 

C. SELECTING FIELD TESTERS TO INTERVIEW 

Application of the Harvard Business School’s Redesigning Product/Service 
Development program field tester selection criteria was relatively straightforward as applied 
to the submarine customer product development interview planning. The first step was to 
define users versus purchasers. For the submarine C4I systems of interest, the users are the 
officers and enlisted communications technicians assigned to operational units whereas the 
purchasers are program office and resource sponsor personnel. As outlined in the L.L. Bean 
method, the product development interviews were limited to the user community. The 
second step in selecting submarine field testers is to define experienced users. In this case, 
the level of experience with the submarine C4I systems of interest was the determining 
factor. Experienced users were defined as those submarine operators who had recently 
returned from a six-month deployment while equipped with those systems. 

Defining demographic categories in which diverse representation among experienced 
users is desired constitutes step three of the field tester selection process. For submarine C4I 
systems, the important demographic diversity consideration is the rank of selected field 
testers. It is desirable to include experienced users from all ranks that operate the equipment 
or depend upon its operation to execute the submarine’s mission. This results in selection of 
a broad cross-section of ranks onboard from the junior enlisted communicators (typically a 
third class petty officer/E-4) to the Commanding Officer (a Commander/0-5). Other 
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possible demographic categories (gender, geographic location, etc.) were determined to be 
unimportant for this study. 

The final step in determining field tester selection criteria is to define common 
characteristics for experienced, demographically diverse users. In this regard, L.L. Bean’s 
set of common characteristics for field testers was directly applicable to the desired 
characteristics of submarine field testers. Specifically, submarine field testers were pursued 
who were articulate, analytical, honest, unbiased and timely in providing feedback. 

In addition to applying the Harvard Business School’s step-by-step method to 
defining submarine field tester selection criteria, two additional considerations specific to the 
submarine community were used to help select and recruit field testers for this study. These 
considerations are discussed below. 

1. Submarine Deployment Cycles 

The typical submarine training and deployment schedule includes eighteen months of 
in port and underway training in the vicinity of the ship’s homeport followed by a six-month 
duration overseas deployment. The crew’s operating expertise in all areas, including 
communications, can be expected to peak during the overseas deployment when the 
operational tempo normally requires the submarine to be underway about 75-80 percent of 
the time. Additionally, the most demanding and intensive communications scenarios occur 
during the overseas deployment. For these reasons, the ideal submarine field testers would 
have recently completed an overseas deployment. 
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2. Submarine Community Input 

In order to ensure that selected submarine field testers would be the most proficient 
and knowledgeable communications personnel available, the submarine force type 
commanders in the Atlantic and Pacific fleets were consulted and asked to provide their 
perspective regarding which submarines had consistently demonstrated the highest level of 
communications expertise. Since the type commander personnel have detailed knowledge of 
the communications proficiency for all submarines under their command, this input was 
heavily weighted in selecting the submarine field testers for this study. 

Applying the tenets of the Harvard Business School process and including inputs 
from submarine force operational commanders, four submarines were selected for this study: 
USS JEFFERSON CITY (SSN 759), USS ASHEVIFFE (SSN 758), USS SANTA FE (SSN 
763) and USS OFYMPIA (SSN 718). Twenty-two submarine field testers were chosen from 
the crews of these ships, including two Commanding Officers, two Operations Officers, four 
Communications Officers, four Communications Feading Chief Petty Officers, five 
Communications Watch Supervisors and eight Communications Watch Standers. 

D. CONDUCT OF INTERVIEWS 

Several key tenets of the Harvard Business School’s Redesigning Product/Service 
Development program to obtain the customer voice describe attributes that are specific to the 
conduct of the interview itself. The following paragraphs describe how the tenets of “Go 
right to your customers turf’, “Fisten to your customers face-to-face” and “Watch your 
customers actually using your products or services” were applied during the submarine 
communications product development customer interviews (Garvin, 2000, p. 11). 
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1 . 


Interview Locations 


For the submarine community, the customer’s turf is easily defined as the submarine 
itself. All submarine customer product development interviews for this study were 
conducted face-to-face with selected field testers onboard operational fast attack submarines. 
Interestingly however, the different demographic groups (officer and enlisted personnel) 
require slightly different locations onboard the submarine to feel more at ease during the 
interviews. For officers, the interview location chosen is the submarine wardroom where the 
officers typically meet for meals and training evolutions on an underway submarine. The 
wardroom offers a relaxed atmosphere where the officers can comfortably describe their 
experiences with the C4I systems of interest to this study. For enlisted communications 
technicians, the radio room offers the best area to interface with the selected field testers as it 
makes them feel at ease in their working environment and affords the opportunity for them to 
demonstrate the operation of the products during the conduct of the interview (another key 
tenet of the L.L. Bean method). All of the submarine C4I systems of interest to this study are 
physically located in the submarine’s radio room. 

2. Interview Schedule 

The Harvard Business School’s process for interviewing field testers requires that the 
interviews take no more than one hour of the field tester’s time. This helps ensure that the 
field testers enjoy the process and don’t feel burdened by it. The L.L. Bean method relies on 
a small group of motivated field testers to be used again and again to help design new 
products and improve existing ones. Field testers should perceive the interviews as an 
opportunity to contribute their ideas to the development of superior products rather than just 
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another task to add to their already lengthy daily routine. It is particularly important to 
follow this guideline when working with submarine field testers since they are extremely 
busy with the day-to-day operations of the submarine. All of the interviews planned for this 
study were scheduled for a one-hour duration in keeping with the Redesigning 
Product/Service Development program criteria and to limit the impact on the submarine’s in 
port schedule. 

E. CHAPTER SUMMARY 

This chapter has described three main areas of research methodology required in this 
thesis. The first area involved planning the product development interviews to include 
determining the composition of the interview team and developing the questions to be used 
during the interview. The second area consisted of selecting field testers from the submarine 
community to interview. Finally, the third area was the conduct of the interviews 
themselves. The results of these interviews will be critical in determining the applicability of 
the Harvard Business School’s Redesigning Product/Service Development program to 
military C4I systems development and to develop useful product requirements for the 
submarine C4I systems of interest. The following chapters analyze and summarize the 
results of these submarine customer product development interviews. 
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IV. DATA ANALYSIS AND INTERVIEW RESULTS 


A. INTRODUCTION 

This chapter analyzes interview responses from submarine field testers. Selected 
submarine crew members answered five open-ended questions regarding the submarine C4I 
systems of interest and provided recommendations for their improvement. The interview 
results were analyzed in accordance with the Harvard Business School’s Redesigning 
Product/Service Development method. This chapter discusses the post- interview debriefings 
conducted immediately following each series of interviews and describes how submarine 
field tester interview results are translated into summarized quotations and descriptions. 
These summarized results are then translated into product requirements and design ideas 
following the L.L. Bean model. 

B. INTERVIEW RESULTS 

The data for this thesis are the submarine field tester comments recorded during one- 
on-one interviews with them. Twenty-two carefully selected submarine crew members were 
interviewed using the L.L Bean product development interview techniques as described 
above. Interviewee responses were recorded exactly without interpretation, filtering or 
summary as recommended in the Harvard Business School’s Redesigning Eroduct/Service 
Development Toolkit (Garvin, 2000). Post-interview debriefings were conducted as soon as 
possible following each series of interviews, and follow-up meetings were held to summarize 
and translate submarine field tester interview results into product requirements and design 
ideas. 
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1. Post-Interview Debriefings 

The Harvard Business School program suggests that the interviewer and recorder 
conduct post-interview debriefings with one another as soon as possible following each 
interview. This allows the team to review the interview notes and fill in any gaps while the 
interview is still fresh on their minds. The objective of the debriefings was to capture what 
the submarine field testers thought and felt most deeply about the submarine C4I systems of 
interest by selecting the most evocative quotations and vivid descriptions they provided 
during the interview. Examples of these quotations regarding the submarine C4I systems of 
interest include: 

“Too many things that we are getting -- a bunch of geek toys -- do we really 
need them all?” 

“The installation process -- they install equipment, they give no tech support, 
they give us on-the-fly training, they say good bye and have fun. Then, the 
equipment breaks down and sits idle for the rest of the deployment.” 

“Biggest problem is lack of time for training -- knowledge not used is 
knowledge not gained.” 

“All of this shit is hard to fix - none of it is easy!” 

Appendix B provides complete submarine customer product development interview 
debriefing results arranged by interview question and submarine C4I system of interest. 

2. Summarizing field tester Quotations and Descriptions 

In accordance with the Harvard Business School method, submarine field tester 
interviews are summarized and translated in three separate phases. The first phase involves 
summarizing and pooling field tester quotations and descriptions into general categories. As 
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an initial step, quotations and descriptions are pooled according to the interview question and 
system of interest to which they apply. These pooled quotations and descriptions are then 
prioritized and reduced according to the interview team’s understanding of field tester 
priorities. Any questions regarding field tester priorities are resolved by referring to the 
interview transcripts. This process continues until the quotations and descriptions are 
reduced to a manageable number, approximately three to five times the number of team 
members as defined by the Redesigning Product/Service Development Toolkit (Garvin, 
2000). Next, prioritized quotations and descriptions are rearranged into clusters by themes, 
ignoring the interview questions. Any quotations/descriptions not grouped through this 
process are evaluated separately according to their perceived importance, referring to the 
interview transcript as required. These ungrouped quotations/descriptions are then either 
eliminated or if deemed important enough, they each become a separate group of one. 
Finally, each cluster of submarine field tester quotations and descriptions are summarized 
with a brief phrase or statement. 

For each of the submarine C4I systems of interest, application of this process led to 
the following field tester quotation/description summary statements: 
a. UHF MDR Capability 

1) Only One User Allowed at a Time 

2) Inadequate Operator Training 

3) System Too Slow 

4) Spare Parts Support Inadequate 

5) Poor Installation Planning 

6) No Line of Sight Internet Protocol Communications Capability 
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b. Mini-DAMA UHF Transceiver: 


1) Spare Parts Support Inadequate 

2) Inadequate Operator Training 

3) Insujficiently Strong Connectors Used 

4) Proliferation of Laptop Computers in Radio Room is a Problem 

5) Poor Backwards Compatibility with Earlier Systems 
c. Submarine Tactical Data Links: 

1) Systems Not Operationally Useful 

2) Inadequate Operator Training/Technical Support 

3) Inadequate Stowage Available for Hard Copy Technical Manuals 
Recurrence of summary statements regarding inadequate spare parts support and 

operator training over multiple systems is likely indicative of a systemic problem with 
submarine C4I system development. Appendix C provides the complete listing of 
summarized grouped quotations and descriptions developed from the submarine product 
development interviews as part of this study. 

3. Translating Summarized Quotations and Descriptions into Product 

Requirements 

The second phase of the Harvard Business School’s Redesigning Product/Service 
Development method to translate field tester interview results into design ideas involves 
developing product requirements from the summarized field tester quotations and 
descriptions. First, each quotation/description summary statement developed above is 
translated into one or more product requirements. These product requirements are then 
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prioritized and reduced using a process similar to that used to prioritize and reduce submarine 
field tester quotations and descriptions. Questions regarding field tester priorities are 
resolved by referring to interview transcripts or quotation/description sheets. In this way, 
product requirements are reduced to a manageable number of approximately three to five 
times the number of team members (Garvin, 2000). Prioritized requirements are then 
grouped into clusters by themes and any ungrouped requirements are evaluated for 
elimination or retention based on their importance, referring to quotation/description sheets 
as necessary. Requirements are reviewed to ensure submarine field tester priorities are 
accurately captured and no redundancies exist until only essential requirements remain. 

For each of the submarine C4I systems of interest, application of this process led to 
the following product requirements: 

a. UHF MDR Capability 

1. System shall allow for multiple simultaneous users. 

2. Operator training shall address all facets of system operation. 

3. Operator training shall be made available to all planned operators. 

4. System shall allow for higher data throughputs. 

5. System sparing shall provide adequate support for deployed operations. 

6. System installations shall be planned to minimize submarine schedule 
impacts. 

7. System shall provide a Line of Sight Internet Protocol communications 
capability. 

b. Mini-DAMA UHF Transceiver 

1) System sparing shall provide adequate support for deployed operations. 
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2) Operator training shall address all facets of system operation. 

3) Operator training shall be made available to all planned operators. 

4) System connectors shall withstand normal wear and tear to include routine 
maintenance requirements. 

5) Operating workstation for the system shall be environmentally qualified with 
respect to anticipated shock and vibration conditions. 

6) Operating workstations in the Radio Room shall be consolidated to the 
maximum extent possible. 

7) System shall be interoperable with UHF Transceivers in service on other 
Navy units. 

c. Submarine Tactical Data Links 

1) Systems fielded shall be operationally useful for submarines. 

2) Non-op e rationally useful systems shall be removed from submarine Radio 
Rooms. 

3) Operator training shall address all facets of system operation. 

4) Operator training shall be made available to all planned operators. 

5) Adequate stowage shall be provided for all required hard copy technical 
manuals and/or technical manuals shall be provided in electronic format. 

Requirements for improved and more readily available training flow naturally from 
weaknesses identified in all three submarine C4I systems of interest. Improved logistics 
support is also a recurring requirement across multiple systems. Appendix D presents the 
complete product requirements listing developed for the submarine C4I systems of interest as 


part of this study. 
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4. Translating Product Requirements into Design Ideas 

The final phase of summarizing and translating field tester interviews in accordance 
with the L.L. Bean method begins with brainstorming to produce many product design ideas 
for each requirement developed in the previous phase. To encourage innovation, no design 
idea should be considered too “crazy” for consideration. Generated design ideas are then 
pooled and reduced in a process similar to that used for field tester quotation/descriptions and 
product requirements. Design ideas are then grouped into clusters by themes, and any ideas 
not grouped are evaluated for elimination based on their ability to meet the product 
requirements. The resulting design idea listing is compared to the submarine field tester 
quotation/description sheets to ensure that the ideas and concerns of the experienced, 
demographic ally diverse product users are driving the design idea development (Garvin, 
2000 ). 

For each of the submarine C4I systems of interest, application of this process led to 
the following design ideas: 

a. UHF MDR Capability 

1) Multi-User UHF MDR Network - software and hardware upgrades to allow 
more than one submarine user at a time. 

2) Mobile Training Team - Team of system experts dispatched to submarines 
soon after upgrades installed to provide comprehensive training (classroom 
and underway operational training). 

3) Advanced Digital Waveform - increase system throughput. 

4) Improved logistics planning - enhance spares support. 
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5 ) Submarine Installation Planning Team - gather installation lessons learned 
and improve planning process. 

6) Battle Force E-mail System - Add this Government Off-The-Shelf (GOES) 
system to submarine Radio Rooms to provide a Line of Sight Internet Protocol 
communications capability. 

b. Mini-DAMA UHF Transceiver 

1) Mobile Training Team - Team of system experts dispatched to submarines 
soon after upgrades installed to provide comprehensive training (classroom 
and underway operational training). 

2) Improved logistics planning - enhance spares support. 

3) Stronger system connectors - upgraded connectors to provide added strength. 

4) Rugged, consolidated workstations - eliminate problems with laptops and 
reduce number of workstations. 

5) Standard Operating Procedure improvements - describe equipment setups 
required to ensure interoperability with other in service UHF transceivers. 

c. Submarine Tactical Data Links 

1) Common Operating Picture (COP) - new submarine tactical data transfer 
paradigm required as current legacy systems are clearly not useful in today’s 
submarine operations. 

2) Mobile Training Team - Team of system experts dispatched to submarines 
soon after upgrades installed to provide comprehensive training (classroom 
and underway operational training). 
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3) Interactive Electronic Technical Manuals (lETMs) - convert hard copy 
technical manuals to electronic format. 

Mobile Training Team visits could enhance operator training for all submarine C4I 
systems of interest. Improved logistics planning would also provide significant benefits 
across multiple systems. Appendix E lists all submarine C4I system of interest design ideas 
developed as a result of this study. 

E. CHAPTER SUMMARY 

The Harvard Business School’s Redesigning Product/Service Development process 
has been applied to develop new design ideas for submarine C4I systems of interest. This 
chapter described the phases involved in the application of this method to translate submarine 
field tester interview results into product requirements and design ideas. These design ideas 
can be used to improve the C4I systems of interest and enhance the development of new 
submarine C4I systems with similar requirements. The following chapters provide lessons 
learned and conclusions regarding the application of the Harvard Business School’s 
methodology to military C4I system development and give recommendations for future 
research. 
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V. EVALUATION OF HARVARD BUSINESS SCHOOL PROGRAM APPLIED TO 
MILITARY C4I SYSTEMS DEVELOPMENT 

A. INTRODUCTION 

The preceding chapters have detailed an application of the Harvard Business School’s 
Redesigning Product/Service Development program to obtain customer inputs for submarine 
C4I systems development. This chapter will identify lessons learned through applying the 
L.L. Bean method for developing new product ideas to submarine systems and generalize 
these lessons learned to other military systems. Benefits and drawbacks of this method for 
obtaining operating forces’ input to C4I systems development will be identified and obstacles 
to employing these methods in the military services will be explored. Conclusions and 
recommendations for further research will then be provided in the final chapter of this paper. 

B. BENEFITS OF L.L BEAN MODEL 

The greatest advantage of the Harvard Business School’s Redesigning 
Product/Service Development model as applied to military C4I systems development is its 
focus on obtaining product design ideas from the true experts in the field, those that actually 
use the systems to accomplish their assigned missions. By providing a logical and 
comprehensive framework for capturing the experiences (both good and bad) of the actual 
customers, the Harvard Business School method enables system developers to capture the 
true mission need that the system should fulfill and significantly enhances the developers’ 
understanding of the system’s operational requirements. By immersing product developers 
in their customers’ experiences through face-to-face interviews on the customer’s turf with 
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an opportunity to watch customers actually using the C4I systems of interest, the L.L. Bean 
method enhances the intensity of the developer and customer interactions. Gruner and 
Homburg (2000) have developed a theoretical justification for the positive influence of 
customer interaction in the product development process using resource dependence theory, 
and they empirically supported their theory that the intensity of customer interaction in the 
product development process positively impacts product success through intensive study of 
the German machinery industry. Their research also suggests that customer interaction is 
most effective in developing products where a high degree of innovation is required and 
where the customers have a high level of expertise with respect to the product. Both of these 
criteria apply strongly to military C4I systems, indicating the importance of involving 
operational users in the product development process and implying that the customer focus 
inherent in the L.L. Bean method should lead to product development success. 

Another important benefit of the Harvard Business School’s Redesigning 
Product/Service Development program is their methodical approach to identifying customers 
with an optimal mix of expertise and experience to serve as field testers and provide the 
necessary user insights into the development process. Gruner and Homburg’s (2000) 
research also supports the hypothesis that the characteristics of the involved customers have a 
direct impact on product development success. By providing a detailed framework for 
identifying experienced, demographically diverse users to participate in the product 
development process, the Harvard Business School method maximizes the potential for a 
successful outcome. Figure 3 graphically depicts the conceptual framework of Gruner and 
Homburg’s findings with respect to customer interaction and new product success. The L.L. 
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Bean method integrates both criteria identified by Gruner and Homburg as important to 


successful product development. 



Figure 3: Conceptual Framework for Customer Interaction and New Product Success 

(From: Gruner and Homburg, 2000) 


A final important benefit of the Harvard Business School program is the step-by-step 
methodology provided to translate customer interview results into essential product 
requirements and develop design ideas to precisely meet those identified requirements. Each 
step of this process emphasizes comparison of the result with customer statements of 
necessary product features to ensure that the ideas and concerns of experienced, 
demographically diverse product users are continuing to drive the development process. This 
step by step process to capture customer mission needs and product requirements lends itself 
well to submarine C4I systems development and helps to ensure that successful outcomes 
will be repeatable over a wide variety of military C4I systems applications. 
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C. LIMITATIONS OF L.L. BEAN MODEL 

A significant limitation of the Harvard Business School’s Redesigning 
Product/Service Development program is that it requires that experienced customers exist 
from which to obtain essential product requirements and stimulate design ideas. While this is 
possible in most product development applications, highly innovative product development 
efforts may have difficulty identifying field testers to interview. For example, the developer 
of the first U.S. Navy submarine, John P. Holland, would have encountered significant 
difficulty in identifying experienced submarine users to participate in his product 
development process. Thus, there likely exists an upper limit to the capacity of the L.L. Bean 
method’s ability to provide useful input to highly innovative military C4I systems 
development efforts. 

Another limitation of this method is that it addresses customer interaction with 
product developers only in the early design/concept idea stage of product development. 
Gruner and Homburg’s (2000) research results encourage interaction in the late stages of the 
product development process (i.e., prototyping and product launch) as well as the early 
stages. This suggests that other methods in addition to the Harvard Business School program 
are required to maximize the benefits of customer interaction during the product development 
process. 

C. OBSTACLES TO IMPLEMENTING THE L.L. BEAN MODEL 

The most significant obstacle to effective implementation of the Harvard Business 
School’s Redesigning Product/Service Development program in military C4I systems 
development is the DoD acquisition model itself. The L.L. Bean method is premised upon 
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direct interaction between experienced users and system developers. Although the DoD 
acquisition model encourages and requires acquisition and requirements community 
personnel to interact with the user community to enhance new product developments, it 
makes no specific provisions for the actual system developers (i.e., defense contractors) to 
interact on a systematic basis with operational users, particularly in the early stages of 
concept development. This places the burden for capturing and translating operational user 
needs and requirements on the acquisition community and risks inaccurately or inadequately 
conveying the user’s intent to the actual system developers. Fortunately, program managers 
have significant leeway in implementing DoD regulations, and many have already begun to 
realize the significant advanta^s that can accrue from promoting direct interaction between 
operational users and system developers. For example, the Submarine Communications 
Program Office is effectively applying this technique to enhance the development of the 
VIRGINIA (SSN 774) Exterior Communications System by periodically sending submarine 
communications electronics technicians assigned to operational fleet units to the system 
developer’s integration facility, allowing contractor system and software engineers to better 
understand the operator perspective and integrate it successfully into the final product. 

Other less significant obstacles to effectively implementing the Harvard Business 
School method in military C4I systems development include funding constraints (both the 
funds required to implement the method as well as funds required to implement design ideas 
identified through the process) and the training required to teach the method to appropriate 
program office and system developer personnel. Efficient budgeting and planning practices 
by the program office and contractor facility are required to overcome these lesser obstacles. 
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E. CHAPTER SUMMARY 

Benefits, limitations and obstacles to implementing the Harvard Business School’s 
Redesigning Product/Service Development process have been outlined in this chapter. The 
L.L. Bean method effectively applies the tenets of intensity of customer interaction and 
characteristics of the involved customers researched by Gruner and Homburg. However, this 
method focuses solely on the early stages of the product development process, and it focuses 
on direct interaction between operational users and system developers that can be harder to 
achieve in military systems development. The final chapter provides conclusions regarding 
the application of the Harvard Business School’s methodology to military C4I system 
development and gives recommendations for future research. 
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VI. CONCLUSION AND RECOMMENDATIONS 


A. CONCLUSION 

This research applied a commercial product development process developed by the 
Harvard Business School and L.L. Bean, Incorporated to military C4I systems development. 
The redesigning product/service development program developed by Garvin focuses on 
interviews of carefully selected, experienced product users in order to better understand the 
voice of the customer and derive new product requirements and design ideas. Open-ended 
questions are developed and asked in the users’ environment to encourage them to tell a story 
that captures the essence of their product needs. Interview results are analyzed and 
congealed into product requirements and design ideas through step-by-step review and 
analysis. This thesis used this method to capture the voice of the submarine customer as 
applied to communications, command and control systems. 

Research for this thesis included a review of current best commercial practices in 
injecting customer input into the new product development process. Previous research on 
applying commercial product development processes to military systems development was 
also reviewed. In order to gather submariner input, twenty-two officers and enlisted 
personnel currently assigned to operational submarines were selected as C4I system field 
testers. These experienced users were interviewed onboard their ships to capture their 
perspective on system requirements and ideas for improvement. 

The interview results were translated into concrete product requirements and design 
ideas using the step-by-step method recommended by the Harvard Business School program. 
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Design ideas developed during the course of this research are currently being implemented to 
improve the effectiveness and user acceptance of submarine C4I systems. Initial results of 
this implementation are promising. The Submarine C4I Mobile Training Team, in particular, 
has demonstrated significant success in increasing the ability of the submarine users to 
operate and maintain new C4I system enhancements effectively. The training team benefited 
greatly from the data collected as part of this thesis, incorporating the customer insights 
directly into their curriculum. Feedback from their subsequent training visits to submarines 
has been overwhelmingly positive. Space and Naval Warfare System Command program 
offices are currently implementing other design ideas developed through this research, and 
it’s still too early to judge their effectiveness in actual submarine force operations. 

In addition to leveraging experienced submarine user operational knowledge to 
develop concrete design ideas for improving submarine C4I systems, the larger benefit of this 
thesis is the successful application of a commercially developed product development 
process in military C4I systems development. The same Harvard Business School program 
that enhances L.L. Bean’s ability to integrate customer input into the development of new 
hunting boots has demonstrated utility in capturing experienced military user’s C4I system 
product requirements and design ideas as well. The basic tenet of the redesigning 
product/service development program holds that the customer, not the product designer, is 
the true system expert. So, it is obviously important to make every effort to accurately 
capture the voice of the customer early and often in the new product development process. 
The DoD acquisition system also recognizes the importance of customer (in this case, the 
warfighter) input into the new product development process. The Mission Needs Statement, 


developed by the operating forces, marks the initiation of any new system development. 
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However, as many successful commercial companies have come to realize and the DoD is 
beginning to understand, a Mission Needs Statement alone is an inadequate vehicle for 
completely capturing the voice of the customer. A continuing dialogue between the user, 
requirements and acquisition communities is required to ensure that system developers truly 
understand the customer’s needs and priorities. Further, the defense contractors who actually 
develop military systems need to be included in this dialogue. This will become increasingly 
important as we strive to transform the military and our acquisition processes. Admiral 
Dennis Blair, Commander-in-Chief of the Pacific Command, also espouses this view, 
labeling it “Acquisition by Adaptation”: 

The paradigm for the future should be based on quicker partnerships ... 
putting a prototype system out quickly, then adapting and improving it as it’s 
fielded. The only way to do this is to connect the engineers directly with the 
fleet and field units, with the acquisition community as enablers for that 
process, not controllers of it, and the CINC’s identifying requirements, setting 
priorities and providing venue for systems development (Blair, 2001, p. 4). 

The Submarine Communications Program Office is embracing this vision by pursuing 
innovative ways to inject the voice of the submarine warfighter into the development of the 
VIRGINIA (SSN 774) Exterior Communications System, maximizing the direct interaction 
of submarine communications technicians assigned to operational fleet units and system 
developers employed by Lockheed Martin under contract to deliver the system. As an 
outgrowth of the concepts developed during the course of this thesis, a comprehensive Fleet 
Operator System Development and Integration Assistance Team has been chartered to 
directly connect experienced submarine communicators with the engineers developing the 
next generation of submarine communications systems. Qualified communications 
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watchstanders and maintenance technicians from afloat units identified by the Commander, 
Naval Submarine Forces in Norfolk, Virginia will periodically travel to the Lockheed Martin 
Exterior Communication Systems integration facility in Eagan, Minnesota to participate 
directly in system development and integration. By placing aibmarine communications 
system experts from the user community on site with system developers, this initiative should 
help ensure that the voice of the submarine warfighter is well represented in the final Exterior 
Communications System delivered to the Navy and enhance overall submarine customer 
satisfaction. 


B. RECOMMENDATIONS FOR FUTURE RESEARCH 

The Harvard Business School’s Redesigning Product/Service Development program 
is only one example of a commercial product development process that can be successfully 
adapted to identify and inject the voice of the warfighter into military C4I systems 
development. In support of the DoD’s continuing quest for acquisition excellence, this thesis 
provides five areas to be considered for future research. 

1. Investigate Web-based methods for capturing the voice of the warfighter. 

Drazen Prelac’s Information Pump (2001) can provide the voice of the customer 
via the Web and enable customers to compare and contrast desirable product 
features via web-based interaction with one another. This method also has the 
distinct advantage of incorporating a scoring method for qualitative and 
quantitative feedback. This avenue of research is worth pursuing because it may 
capture the voice of the warfighter more efficiently and with less cost as 
compared to the Harvard Business School’s Redesigning Product/Service 
Development method. 

2. Explore new methods for increasing direct interface between military system 
users (warfighters) and product developers (defense contractor engineers). 

Benefits from this research would be two-fold. Product developers would gain a 
better understanding of user needs and requirements, while warfighters would 
simultaneously gain a better appreciation for military system capabilities and 
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limitations. These methods should therefore lead to better military products that 
are more effectively employed in the field. 

3. Conduct a quantitative analysis of the tangible benefits of injecting the voice 
of the warfighter directly into the new product development process. This 
study would entail a comparison of military product developments that used a 
high degree of experienced user input during the development process with those 
that involved warfighters to a lesser degree. Quantitative metrics involving 
specific performance achievements and cost reductions would be developed and 
analyzed. 

4. Create a database of commercial best practices that have been proven to 
enhance military systems product development. The product of this research 
would become a toolkit allowing government program managers to select 
commercial techniques that have demonstrated success in military applications. 

5. Apply the Harvard Business School’s Redesigning Product/Service 
Development program methods to the development of an Operational 
Requirements Document. This research would apply the same method 
described in this thesis to a new start military program at the outset to determine if 
the same type of one-on-one interview process with experienced users can be 
useful to formulate an Operational Requirements Document for a previously non¬ 
existing capability. 

Military systems’ development can be greatly enhanced by more effectively 
communicating warfighter requirements to the engineers tasked with designing and building 
them. Commercial methods for capturing the voice of the customer and ingraining it into the 
product development process can prove helpful in military applications. Government 
program managers should strive to increase the exchange of ideas between their warfighter 
customers and the engineers actually developing the systems under their cognizance. 
Commercial best practices such as the Harvard Business School’s Redesigning 
Product/Service Development program can help them do it. 
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APPENDIX A. PRODUCT DEVELOPMENT INTERVIEW DATA FORMS 


a. Describe the most 
demanding communications 
period that occurred during 
the deployment and how well 
or how poorly the new C4I 
system installations supported 
the ship during that period? 

b. Describe the most difficult 
maintenance evolution 
involving the new C4I 
systems that occurred during 
the deployment and how well 
or how poorly the technical 
documentation supported your 
maintenance efforts. 

c. Describe the training 
provided during the 
installation of the new C4I 
systems and how well or how 
poorly that training prepared 
you for operating and 
maintaining the equipment 
during your deployment. 

d. Describe the impact of the 
new C4I system installation 
on the ship’s routine/schedule. 

e. Are there any other issues 
or problems with the new C4I 
systems? Were any Casualty 
Reports (CASREPS) 
involving the new C4I 
systems required during the 
deployment? 
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APPENDIX B. PRODUCT DEVELOPMENT INTERVIEW RESULTS 


a. Describe the most demanding communications period that occurred during 
the deployment and how well or how poorly the new C4I system installations 
supported the ship during that period? _ 

UHF MDR Capability: 

SIPRNET to CTF 12 ASW WECAN limited by satellite time-sharing with other 
submarine, time broken by odd hour and even hour. WECAN used by a lot of 
players; function very good - biggest drawback was we had to share. 

Continually questioned whether or not the data got off the ship; an example was when 
we had 8 email messages received we had no indication if the messages were sent 
really transmitted. 

Server at SUBPAC was a challenge; MIESTAR 1 & 2 great reach back to SUBPAC; 
server there can only support 1 user at a time; degraded capability of the technology 
(UHF MDR). 

Major challenge was that we had no one trained on the exchange server - only one 
sailor had the training, and he left the ship due to a family emergency. 

Time sharing a challenge - cannot always connect to shore server when desired. 

UHF MDR Capability with the CHAT software is a neat system. 

Convenient way to make required reports. 

Biggest limitation is that only one submarine can use it at a time, during one period 
the SANTA EE used it for 8-10 hours; it appears it is first come, first serve. 

So slow - requires too much periscope depth time -extra time at comms depth means 
mast exposure time is increased. 

Sent overlays this way during COMPTUEX -- had to share time with other 
submarines. Great way to send plans and overlays. 

Timeframe for the use of UHF MDR Capability controlled by SUBPAC 
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a. Describe the most demanding communications period that occurred during 
the deployment and how well or how poorly the new C4I system installations 

supported the ship during that period? _ 

Mini-DAMA UHF Transceiver: 

Eased operability of Radio Room equipment - less physical twisting and turning. 

The biggest problem was the interface with the STENNIS whose crew was relatively 
new and had not operated with a submarine outfitted with Mini-Dama. They were 
not familiar with the dual WSC 3 challenge. In Singapore finally figured out the 
difficulty and then we had the STENNIS make the proper adjustments. Recommend 
that the Sub Liaison Officer be given good briefing on Mini-DAMAAVSC-3 
problems; the people on the CV need education of the capability of the equipment on 
the submarine; need configuration lineups shared with the BG. 

Mini-DAMA was slapped on in a day, had minima! training during the installation. 

Mini-DAMA spare cards among all the submarines is inconsistent - no standard spare 
equipment given with the installation; no standard ready spares; lost a power supply 
and did not have one to replace it. 

Mini-DAMA - great system! 

I wish I had DAMA in an all in 1 package (referring to satellite channels assigned). 
Mini DAMA Line of Sight communications are limited in range for UHE Link 11. 
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a. Describe the most demanding communications period that occurred during 
the deployment and how well or how poorly the new C4I system installations 
supported the ship during that period? _ 

Submarine Tactical Data Links: 

Did not utilize S-TADIL-J very well at all - never used it. 

S-TADIL-J - Computer glitch caused BG Computer to go down. Training & Tech 
support not good - immature system. The idea that this new technology needs to be 
put in the Fleet quickly with no manuals or tech support or training is flawed. 

Had on-the-job training for 2 or 3 days during the Pre-Overseas Movement cycle - 
this was 5 months prior to deployment so everything the sailors learned was lost. We 
had very little documentation and the installation was the week before the 
deployment. 

S-TADIL-J - Installed at the last minute; tested the system during installation with 
the shore rather than out at sea with another submarine or even with CV. This is did 
not help us at all. Spent countless days, man-hours and contributed to intense 
frustration trying to get the problem resolved with NCTAMS (sub rate is 19.4) as it 
turned out all it took was turning a switch at the shore site. No SOP written for the 
shore or CV to work with the sub with this equipment. Worthless system. 

LINK 16 - we never use it. It stays off, it is like a big heater; did not use the entire 
deployment; BG needs it but never used (with us) it either, did not get any training on 
it. Only good thing about it, it gave the boat an additional handset in radio; it also 
grounded out the BRA 34 antenna. 

LINK 11 - Never used it either. 

Subtract some equipment from the radio room that we do not use. 

S-TADIL-J - the hard drives never worked so we never operationally used it on the 
deployment. 
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b. Describe the most difficult maintenance evolution involving the new C4I 
systems that occurred during the deployment and how well or how poorly the 
technical documentation supported your maintenance efforts. _ 

UHF MDR Capability: 

Temporary Alterations - no support for them - when you lose the equipment you 
lose it for good and there is nothing in place to support the equipment. 

Laptops - If you lose it UHF MDR Capability is gone - need bracket to hold it. 
Cable plug in (RJ45) have to plug into the computer is prone to breakage and is 
not supported by onboard spares. 

Need Intra-BG IP connectivity capability! - Look at LOS IP Capability in addition 
to UHF MDR Capability 


Mini-DAMA UHF Transceiver: 

No problems; Tech manual tells what to do. 

Hard to get parts - lots of cannibalizations from ship-to-ship required. 

Submarine Tactical Data Links: 

No space to store manuals - need to have technical manuals published on CD- 
ROM. 

All of this shit is hard to fix - none of it is easy! 

All equipment is too hard to fix; we have two new First Class, the junior sailors 
have no idea about the technical information; do not have time to send all sailors 
to school with all the installations in the schedule. 
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c. Describe the training provided during the installation of the new C4I 
systems and how well or how poorly that training prepared you for operating 
and maintaining the equipment during your deployment. _ 

UHF MDR Capability: 

Insufficient training - only one person was taught, and he has transferred from 
ship. 

Training is effective only if it is on-the-air training, especially for UHF MDR. 

The training classes that we did have provided good baseline understanding for 
networking components. 

Underway training better than in port training, but operational issues can impact 
training. 

Good training, but a little too in-depth. 

The information was presented well since those that did attend were able provide 
the information to others. 

Training is beneficial to the division but when are going to do it? 

Biggest problem is lack of time for training. 

Mini-DAMA UHF Transceiver: 

Need practical training - especially on-the-air formal training. 

The operators need to see how it really works; if we are not able to use with other 
ships, we can’t see if our transmissions are correct or if the equipment is operating 
the right way. 

Hands on training is good, actually operating the equipment is better. 

Hands-on on-the-air operation necessary to really learn. 

Good operator training conducted by SSC-Charleston (“I would not have been 
able to operate and maintain Mini-DAMA if I hadn’t gone to Charleston”) 

Big difference between great training next to pier and underway - operating with 
the equipment is the absolute best training possible. 
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c. Describe the training provided during the installation of the new C4I 
systems and how well or how poorly that training prepared you for operating 
and maintaining the equipment during your deployment._ 


Submarine Tactical Data Links: 

S-TADIL-J - No training provided. 

Need another way to check what we are doing (e.g. after a transmission contact 
the other unit and say “these are my settings what are yours?). Really need other 
ships’ support to do effective training. 

S-TADIL-J - Had Saturday morning training; no operator training; the training 
only dealt with how to load crypto. 

S-TADIL-J - 2 hours training, I think, right after it was installed, it has not been 
turned since then - why do we have it in our radio room? 

Knowledge not used is knowledge not gained. 
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d. Describe the impact of the new C4I system installation on the ship’s 
routine/schedule. _ 

UHF MDR Capability: 

Need to front- load installations during upkeeps to the maximum extent possible. 
Mini-DAMA UHF Transceiver: 

Entire installation was challenging. 

Never open front panel door - no reason to open door. Connectors inside are very 
fragile and if you open door, there is a good chance that you will mess them up. 

Installation occurred during SRA, shipyard routine worked well with the 
installation, it was a good time to do it. 

Submarine Tactical Data Links: 

No installation issues. 
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e. Are there any other issues or problems with the new C4I systems? were 
any Casualty Reports (CASREPS) involving the new C4I systems required 
during the deployment? _ 

UHF MDR Capahility: 

UHF MDR capability was completely successful. 

Data Replication took too long; had to wait until replication is done before 
bringing up UHF MDR Chat, once you start replication you can’t do anything else 
- replicate takes about hour to an hour and a half. 

Too many things that we are getting --a bunch of geek toys -- do we really need 
them all? 

Mini-DAMA UHF Transceiver: 

Mini-DAM A: interoperability challenge is the transfer of information between the 
WSC3 Y2fV3 and Mini-DAMA; Have to have 2 WSC3 on DAMA; need to find 
out what works (equipment) and once it works do not swap it out. 

Laptop Computer is the weakest link for the Mini-DAMA system. 

Most of the problems experienced had to do with the CVN , more training and 
procedures need to be given to the CVN . 

Mini-DAMA has a faulty back plane issue. 

Need a back-up laptop for Mini-DAMA - should be standard parts support. 
Computers should be considered consumables like repair parts. If Mini-DAMA 
computer goes out, you have no spares. If it goes down, you lose capability — 
each boat should a spare. 

Need to consolidate workstations and eliminate proliferation of laptop computers 
in the Radio Room (Note - Ship had installed a flat screen monitor with a 
computer workstation (vice laptop) as the Mini-DAMA controller (replaced a 
broken laptop) for better presentation and a central location of two displays (Mini- 
DAMA and BBS). 
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e. Are there any other issues or problems with the new C4I systems? were 
any Casualty Reports (CASREPS) involving the new C4I systems required 

during the deployment? _ 

Submarine Tactical Data Links: 

S-TADIL-J - Something happened to the hardware outside of the radio room; loss 
of some executable files, no circuit for each, never used the system - tried to use, 
but never could get it to come up. 

LINK 11 - Not used in Arabian Gulf; Receive only when used - rarely 
operational. 

NO STOWAGE - We have no place to store anything in the radio room. Put the 
tech manuals on CD/electronic format that will help alleviate some of the 
problems. 

Stowage is a major issue that no one is addressing. 

S-TADIL-J: Installed, but not tested. 

The process -- they install equipment, they give no tech support, they give us on 
fly training, they say good bye and have fun, the equipment breaks down and it 
sits idle. 
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APPENDIX C. SUMMARIZED FIELD TESTER QUOTATIONS AND DESCRIPTIONS 


UHF MDR Capability; _ 

Summary Statement - Only One User Allowed at a Time 

Time sharing a challenge - cannot always connect to shore server when desired. 

Biggest limitation is that only one submarine can use it at a time, during one period 
the SANTA FE used it for 8-10 hours; it appears it is first come, first serve. 

SIPRNET to CTE 12 ASW WECAN limited by satellite time-sharing with other 
submarine, time broken by odd hour and even hour. WECAN used by a lot of 
players; function very good - biggest drawback was we had to share. 

Timeframe for the use of UHE MDR Capability controlled by SUBPAC 

Server at SUBPAC was a challenge; MIESTAR 1 & 2 great reach back to SUBPAC; 
server there can only support 1 user at a time; degraded capability of the technology 
(UHE MDR). 

Sent overlays this way during COMPTUEX -- had to share time with other 
submarines. Great way to send plans and overlays. 

Summary Statement - Inadequate Operator Training 

Major challenge was that we had no one trained on the exchange server - only one 
sailor had the training, and he left the ship due to a family emergency. 

Insufficient training - only one person was taught, and he has transferred from ship. 

Training is effective only if it is on-the-air training, especially for UHE MDR. 

Underway training better than in port training, but operational issues can impact 
training. 

Good training, but a little too in-depth. 

The information was presented well since those that did attend were able provide the 
information to others. 

Training is beneficial to the division but when are going to do it? 

Biggest problem is lack of time for training. 
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UHF MDR Capability (continued): 


Summary Statement - System Too Slow 

So slow - requires too much periscope depth time -extra time at comms depth means 
mast exposure time is increased. 

Data Replication took too long; had to wait until replication is done before bringing 
up UHF MDR Chat, once you start replication you can’t do anything else - replicate 
takes about hour to an hour and a half. 


Summary Statement - Spare Parts Support Inadequate 

Temporary Alterations - no support for them - when you lose the equipment you lose 
it for good and there is nothing in place to support the equipment. 

Laptops - If you lose it UHF MDR Capability is gone - need bracket to hold it. 

Cable plug in (RJ45) have to plug into the computer is prone to breakage and is not 
supported by onboard spares. 


Summary Statement - Poor Installation Planning 

Too many things that we are getting --a bunch of geek toys -- do we really need them 
all? 

Need to front- load installations during upkeeps to the maximum extent possible. 


Summary Statement-No Line of Sight Internet Protocol Communications Capability 

Need Intra-BG IP connectivity capability! - Look at LOS IP Capability in addition to 
UHF MDR Capability 
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Mini-DAMA UHF Transceiver: 


Summary Statement - Spare Parts Support Inadequate 

Hard to get parts - lots of cannibalizations from ship-to-ship required. 

Mini-DAMA spare cards among all the submarines is inconsistent - no standard spare 
equipment given with the installation; no standard ready spares; lost a power supply 
and did not have one to replace it. 

Need a back-up laptop for Mini-DAMA - should be standard parts support. 

Computers should be considered consumables like repair parts. If Mini-DAMA 
computer goes out, you have no spares. If it goes down, you lose capability — each 
boat should a spare. 

Summary Statement - Inadequate Operator Training 

Mini-DAMA was slapped on in a day, had minimal training during the installation. 

Need practical training - especially on-the-air formal training. 

The operators need to see how it really works; if we are not able to use with other 
ships, we can’t see if our transmissions are correct or if the equipment is operating the 
right way. 

Hands on training is good, actually operating the equipment is better. 

Hands-on on-the-air operation necessary to really learn. 

Good operator training conducted by SSC-Charleston (“I would not have been able to 
operate and maintain Mini-DAMA if I hadn’t gone to Charleston”) 

Big difference between great training next to pier and underway - operating with the 
equipment is the absolute best training possible. 

Summary Statement - Insujficiently Strong Connectors Used 

Never open front panel door - no reason to open door. Connectors inside are very 
fragile and if you open door, there is a good chance that you will mess them up. 

Mini-DAMA has a faulty back plane issue. 
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Mini-DAMA UHF Transceiver (continued): 


Summary Statement - Proliferation of Laptop Computers in Radio Room is a 
Problem 

Laptop Computer is the weakest link for the Mini-DAMA system. 

Need to consolidate workstations and eliminate proliferation of laptop computers in 
the Radio Room (Note - Ship had installed a flat screen monitor with a computer 
workstation (vice laptop) as the Mini-DAMA controller (replaced a broken laptop) 
for better presentation and a central location of two displays (Mini-DAMA and BBS). 


Summary Statement - Poor Backwards Compatibility with Earlier Systems 

The biggest problem was the interface with the STENNIS whose crew was relatively 
new and had not operated with a submarine outfitted with Mini-Dama. They were 
not familiar with the dual WSC 3 challenge. In Singapore finally figured out the 
difficulty and then we had the STENNIS make the proper adjustments. Recommend 
that the Sub Eiaison Officer be given good briefing on Mini-DAMAAVSC-3 
problems; the people on the CV need education of the capability of the equipment on 
the submarine; need configuration lineups shared with the BG. 

Mini-DAM A: interoperability challenge is the transfer of information between the 
WSC3 Y2fV3 and Mini-DAMA; Have to have 2 WSC3 on DAMA; need to find out 
what works (equipment) and once it works do not swap it out. 

Most of the problems experienced had to do with the CVN , more training and 
procedures need to be given to the CVN . 
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Submarine Tactical Data Links: 


Summary Statement - Systems Not Operationally Useful 
Did not utilize S-TADIL-J very well at all - never used it. 

LINK 16 - we never use it. It stays off, it is like a big heater; did not use the entire 
deployment; BG needs it but never used (with us) it either, did not get any training on 
it. Only good thing about it, it gave the boat an additional handset in radio; it also 
grounded out the BRA 34 antenna. 

LINK 11 - Never used it either. 

Subtract some equipment from the radio room that we do not use. 

S-TADIL-J - the hard drives never worked so we never operationally used it on the 
deployment. 

LINK 11 - Not used in Arabian Gulf; Receive only when used - rarely operational. 

Summary Statement - Inadequate Operator Training/Technical Support 

S-TADIL-J - Computer glitch caused BG Computer to go down. Training & Tech 
support not good - immature system. The idea that this new technology needs to be 
put in the Fleet quickly with no manuals or tech support or training is flawed. 

Had on-the-job training for 2 or 3 days during the Pre-Overseas Movement cycle - 
this was 5 months prior to deployment so everything the sailors learned was lost. We 
had very little documentation and the installation was the week before the 
deployment. 

S-TADIL-J - Installed at the last minute; tested the system during installation with 
the shore rather than out at sea with another submarine or even with CV. This is did 
not help us at all. Spent countless days, man-hours and contributed to intense 
frustration trying to get the problem resolved with NCTAMS (sub rate is 19.4) as it 
turned out all it took was turning a switch at the shore site. No SOP written for the 
shore or CV to work with the sub with this equipment. Worthless system. 

No space to store manuals - need to have technical manuals published on CD-ROM. 

All of this shit is hard to fix - none of it is easy! 
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Summary Statement - Inadequate Operator Training/Technical Support (continued) 

All equipment is too hard to fix; we have two new First Class, the junior sailors have 
no idea about the technical information; do not have time to send all sailors to school 
with all the installations in the schedule. 

S-TADIL-J - No training provided. 

Need another way to check what we are doing (e.g. after a transmission contact the 
other unit and say “these are my settings what are yours?). Really need other ships’ 
support to do effective training. 

S-TADIL-J - Had Saturday morning training; no operator training; the training only 
dealt with how to load crypto. 

S-TADIL-J - 2 hours training, I think, right after it was installed, it has not been 
turned since then - why do we have it in our radio room? 

S-TADIL-J - Something happened to the hardware outside of the radio room; loss of 
some executable files, no circuit for each, never used the system - tried to use, but 
never could get it to come up. 

S-TADIL-J: Installed, but not tested. 

The process -- they install equipment, they give no tech support, they give us on fly 
training, they say good bye and have fun, the equipment breaks down and it sits idle. 

Summary Statement - Inadequate Stowage Available for Hard Copy Technical 
Manuals 

NO STOWAGE - We have no place to store anything in the radio room. Put the tech 
manuals on CD/electronic format that will help alleviate some of the problems. 

Stowage is a major issue that no one is addressing. 
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APPENDIX D. SUBMARINE C4I SYSTEM PRODUCT REQUIREMENTS 


UHF MDR Capability: 

1. System shall allow for multiple simultaneous users. 

2. Operator training shall address all facets of system operation. 

3. Operator training shall be made available to all planned operators. 

4. System shall allow for higher data throughputs. 

5. System sparing shall provide adequate support for deployed operations. 

6. System installations shall be planned to minimize submarine schedule impacts. 

7. System shall provide a Line of Sight Internet Protocol communications capability. 


Mini-DAMA UHF Transceiver: 

1. System sparing shall provide adequate support for deployed operations. 

2. Operator training shall address all facets of system operation. 

3. Operator training shall be made available to all planned operators. 

4. System connectors shall withstand normal wear and tear to include routine maintenance 
requirements. 

5. Operating workstation for the system shall be environmentally qualified with respect to 
anticipated shock and vibration conditions. 

6. Operating workstations in the Radio Room shall be consolidated to the maximum extent 
possible. 

7. System shall be interoperable with UHF Transceivers in service on other Navy units. 


Submarine Tactical Data Links: 

1. Systems fielded shall be operationally useful for submarines. 

2. Non-operationally useful systems shall be removed from submarine Radio Rooms. 

3. Operator training shall address all facets of system operation. 

4. Operator training shall be made available to all planned operators. 

5. Adequate stowage shall be provided for all required hard copy technical manuals and/or 
technical manuals shall be provided in electronic format. 
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APPENDIX E. SUMARINE C4I SYSTEM DESIGN IDEAS 


UHF MDR Capability: 

1. Multi-User UHF MDR Network - software and hardware upgrades to allow more than 
one submarine user at a time. 

2. Mobile Training Team - Team of system experts dispatched to submarines soon after 
upgrades installed to provide comprehensive training (classroom and underway 
operational training). 

3. Advanced Digital Waveform - increase system throughput. 

4. Improved logistics planning - enhance spares support. 

5. Submarine Installation Planning Team - gather installation lessons learned and improve 
planning process. 

6. Battle Force E-mail System - Add this Government Off-The-Shelf (GOTS) system to 
submarine Radio Rooms to provide a Line of Sight Internet Protocol communications 
capability. 

Mini-DAMA UHF Transceiver: 

1. Mobile Training Team - Team of system experts dispatched to submarines soon after 
upgrades installed to provide comprehensive training (classroom and underway 
operational training). 

2. Improved logistics planning - enhance spares support. 

3. Stronger system connectors - upgraded connectors to provide added strength. 

4. Rugged, consolidated workstations - eliminate problems with laptops and reduce number 
of workstations. 

5. Standard Operating Procedure improvements - describe equipment setups required to 
ensure interoperability with other in service UHF transceivers 


Submarine Tactical Data Links: 

1. Common Operating Picture (COP) - new submarine tactical data transfer paradigm 
required as current legacy systems are clearly not useful in today’s submarine operations 

2. Mobile Training Team - Team of system experts dispatched to submarines soon after 
upgrades installed to provide comprehensive training (classroom and underway 
operational training). 

3. Interactive Electronic Technical Manuals (lETMs) - convert hard copy technical manuals 
to electronic format 
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